Controlling processing networks

ABSTRACT

According to an embodiment of the invention, a method is provided for allocating load to processing nodes in a processing cluster arranged as an object dependency network. The method includes establishing at a load allocation function a plurality of load allocation function attributes whose values are dependant on the values of attributes of the processing nodes, and receiving requests for processing. The method further includes determining to which of the processing nodes to allocate the requests for processing based on the load allocation function attributes, and allocating requests for processing to the processing nodes in accordance with that determination.

FIELD OF THE INVENTION

This invention relates to controlling processing networks, for exampleto achieve load balancing between multiple processors.

BACKGROUND OF THE INVENTION

Distributed data processing systems are becoming widely used for complexprocessing tasks. By distributing processing between a number ofprocessors such systems are capable of performing complex tasks rapidly.However, to optimise their performance software that is allocated to adistributed system must be split efficiently between the availableprocessors. Some software is specially written for multi-processorsystems and includes instructions indicating which parts of the softwareare to be performed in parallel by which processors. However, this doesnot take account of other tasks that the processors may have tointerleave with the software, and it cannot account for configurationsof processors that were unknown to the software designer or that havearisen because of partial failure or upgrading of a standardmulti-processor system. There is therefore a need for an effective andmore generic system of load balancing.

A sophisticated multi-processor data processing system may be consideredas cluster of processing nodes (CPUs) and a load balancer function. Theload balancer function allocates tasks to the processors according topre-defined rules. When software for providing a certain service is tobe run by the cluster, the processes involved in the software may bedivided so that a number of processing nodes are participating in theproviding of the service in a load sharing fashion. Those processingnodes are termed a load sharing group. The nodes are not restricted toparticipating in the providing of only one service; instead multiplesoftware functions can be allocated to a node. In addition a node willalways be spending some time executing software related to themaintenance of the cluster and the node itself (i.e. the platform).Therefore the processing node requires some processing capacity just toperform its normal maintenance duties.

For each service allocated to a node there will typically be a number ofprocessing entities (processes) executing, each of which provides somepart of the service. In some cases there will even be multiple instancesof the same process executing to increase parallelism and faultisolation.

The overall performance of the cluster is very dependant on theprinciple used by the load balancer to allocate load to the availablenodes. If a node is overloaded then the results of its processing islikely to be delayed. This can be especially serious when those resultsare to be used by another processor because serial delays can then buildup.

SUMMARY OF THE INVENTION

Aspects of the present invention are set out below and in theaccompanying independent claims. Preferred aspects of the invention areset out below and in the dependant claims.

BRIEF DESCRIPTION OF THE DRAWINGS

The present invention will now be described by way of example withreference to the accompanying drawings, in which:

FIG. 1 illustrates the action of dependency in an object dependencynetwork;

FIG. 2 illustrates node, process group and process objects, havingattributes in an object dependency network;

FIG. 3 illustrates correlator, node, process group and process objects,having attributes in an object dependency network;

FIGS. 4 to 7 illustrate the operation of load balancing functions in amulti-processor cluster; and

FIGS. 8 and 9 illustrate the propagation of shutdown-related statusinformation through an object dependency network.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

The following description explains the application of an objectdependency network to implement the controlling function of an adaptiveload balancing function in a multi-processor cluster. This provides afeedback mechanism to the allocation of load.

A state management subsystem (SM) maintains all the managed objects ofthe cluster. Each managed object can have various attributes. Eachattribute is defined by a name and a value. An attribute value caneither be a simple value, or a derived value that is calculated based onsome inputs. The dependencies of a derived attribute value can be takento describe how that value depends on the value of another attributethat is attached to the same managed object or to another managedobject. An attribute value can depend on multiple values and adependency function describes how the value is calculated based on thevalues it depends on. The dependency network automatically invokes thedependency function to recalculate the attribute value when any of thevalues the attribute depends on changes. The managed objects areorganised into a hierarchical network using the order of dependencies oftheir attribute values. This arrangement is illustrated in FIG. 1.

The managed objects maintained in an object dependency network withinthe SM have attributes that correspond to the administrative state,operational state, and usage state defined in the CCITT RecommendationX.731 | International Standard ISO/IEC 10164-2. The value of anadministrative state attribute can set by the operator via an O&Minterface to one of the following: unlocked, shutting down, and locked.An administrative state attribute value set to unlocked means that thesoftware or hardware entity represented by the managed object canperform its normal duties freely. A locked value means that the entityis administratively prohibited to perform its normal duties. A shuttingdown value means that the entity can process whatever ongoing servicerequests it has, but not take on any new work, and when the ongoingservice requests are finished, the administrative state automaticallytransitions to the locked value.

The operational state attribute of a managed object can have either theenabled or disabled value and it is controlled by the system (i.e. theobject itself or the management system by some means, e.g. supervision).An enabled value means that the entity represented by the managed objectis functioning properly and is able to perform its duties normally. Adisabled value means that the entity is not functioning properly and isnot able to perform its duties (i.e. it is considered faulty).

Ultimately the providing of a service can be reduced to the processingof service requests by processes. Each process has the ability to countthe number of service requests it processes, map the number againsttime, and thus construct a service request rate for itself. The servicerequest rate can be expressed as messages per second, transactions persecond, or something similar. In SM each process is represented as amanaged object that has a rate attribute which corresponds to the rateof service requests it is processing and whose value is controlled bythat process itself. This arrangement is illustrated in FIG. 2.

The processes that participate in the providing of a given service on agiven node are grouped into a larger entity that aggregates their work.The service is represented as a process group managed object with itsown aggregate rate attribute in the SM. Dependencies between the processgroup and the processes are defined so that the group determines thataggregate rate attribute by adding together the values of the rateattributes of the processes into a total rate attribute value.

Each node is able to measure the current CPU load that is generated bythe processing of the various service requests it is handling. It can beassumed that an increase in the rate of service requests will eventuallybe reflected as an increase in the CPU load, and a decrease in the rateof service requests will decrease the load. The CPU load can beexpressed as the percentage of CPU cycles that are not allocated to thesystem idle process during a given interval (e.g. over a second). Eachnode is represented in SM as a managed object with a load attribute.

A load balancing function divides the external load coming to thecluster to the load sharing nodes according to a predefined principle.The load balancer can be programmed to give a certain proportion of theexternal load to a given node. This proportion can be expressed with ashare value (W) which can, for instance, be expressed as an integer. Thesum total of share values for all the available nodes (denoted byW_(total)) would then represent the total load that is to be processedby the nodes in the load sharing group.

The dependency network described herein comprises a set of nodes,process groups, and the processes themselves. In applying the objectdependency network to control load balancing it is useful to add acorrelator object whose value depends both on the values of the nodeload attribute and the process group rate attribute. The correlatorobject has a nominal load attribute, a nominal rate attribute, and aload share attribute. The nominal load attribute describes whatpercentage of the CPU should be used in typical load situation. Itshould always be significantly less than 100% so that the system candeal with short bursts of heavy load without problems.

The dependency function of the correlator's load share attribute valueis defined so that it will recalculate the load share value when theobserved load and observed service rates change in the following manner.Let r_(r) be the ratio of observed rate and the nominal rate, and r_(l)the ratio of observed load and the nominal load. Then a delta functionis defined

${r_{l} = \frac{load}{{load}_{n}}};{r_{r} = \frac{rate}{{rate}_{n}}}$${{delta}\left( {r_{r},r_{l}} \right)} = \left\{ \begin{matrix}{{D\text{:}\mspace{14mu} r_{l}} > {1 + ɛ_{high}}} \\{{d\text{:}\mspace{14mu} r_{l}} \leq {1 + {ɛ_{high}\bigwedge r_{r}}} > {1 + \delta_{high}}} \\{{i\text{:}\mspace{14mu} r_{l}} < {1 - {ɛ_{low}\bigwedge 1} - \delta_{low}} \leq r_{r} \leq {1 + {\delta^{\prime}}_{high}}} \\{0\text{:}\mspace{14mu}{otherwise}}\end{matrix} \right.$where D is a large decrease (a predetermined negative number), d is asmall decrease (a predetermined negative number of smaller magnitude), iis an increase (a predetermined positive number), ε_(high) is an upperthreshold for the load, ε_(low) is a lower threshold for the load,δ_(low) is a lower threshold for the rate, and δ_(high) is an upperlimit for the rate. The thresholds and limits can be expressed aspercentages since the ratios are conveniently normalized to one. Otherrationales for calculating whether to apply an increase or decreasecould be employed.

As an alternative, a similar arrangement can be established when theload balancer is not dividing the load based on the nodes, but alsobetween multiple processes in a single node or in multiple nodes. Thebehaviour of the delta function could be modified accordingly, forexample by mapping a proportion of the node's total CPU usage to aprocess group.

Then the load share value at a given time t is calculated using theshare function based on the previous share valueshare(t+1)=share(t)+delta(r _(r) ,r _(l))

${{share}(0)} = \frac{W_{total}}{N}$where N is the number of nodes in the load sharing group and share(0)represents the initial allocation of work to the nodes.

The setup for the operation of this system is illustrated in FIG. 3.

The initial allocation can be made more elaborate if needed. However,the effect of the latter equation is that the sum total of all sharevalues represents the total amount of work that can be allocated to theload sharing group.

The rationale behind the above delta function is as follows. If the loadratio is larger than the upper threshold, the node is overloaded and theload balancer should assign less work for it. Thus the load share valueshould be decreased quite a bit to make a significant reduction in theload. If the load ratio is below the upper threshold, but the rate ratiois above the upper limit, the node is processing more load than allowed,and balancer should assign less work for it, so the load share valueshould be decreased slightly. If the load ratio is below the lowerthreshold, and also the rate ratio between the lower threshold and theupper limit, the node is processing work more efficiently than assumed,and the load balancer can assign more work for it. Thus the load sharevalue can be increased a little. Otherwise the processing of the servicerequests generates the desired load, and the load balancer should keepsending approximately the same amount of work to the node. Thus theshare value should be kept the same. The load share value must then becommunicated to the load balancer at suitable, preferably regular,intervals.

The selection of the D, d, and i values, the thresholds and limits, aswell as the communication interval determine how quickly the loadbalancer will react to the changes in the processing capability of anode. The idea behind the large decrease and small increase is toimplement behaviour similar to the TCP slow start and congestionavoidance algorithms (see IETF RFC 793, “Transport Control Protocol”,September 1981) which will back off rapidly and then increase slowlyuntil the steady state is reached. There should also be lower and upperlimits for the share value that correspond to the maximum and minimumportions of the load that the load balancer can assign to a node. Thevalues can be selected so as to achieve a desired performance.

In comparison with prior art arrangements, this approach offers theadvantage that each correlator can be arranged to recalculate the loadshare value automatically as the observed load and rate values change.Another advantage is that the calculation is based solely on node localinformation, which means that the calculation of the load share valuescan be distributed to each node thus increasing the scalability of theoverall system. Also, the system can allocate a suitable amount of workto the nodes regardless of their processing capacity, thus enabling theload sharing group to be constructed from heterogeneous nodes. Thismeans that it is simple to add a new powerful node to the load sharinggroup, or to allocate some other software functions into an existingnode, and the node will automatically take an appropriate share of theload to itself without the load balancer having to be configured in anelaborate way. Similarly, the arrangement can handle the situation wherea node is withdrawn from a group due to a fault.

The system described above can provide feedback to and can control theload balancing function to adapt the load imposed on individual nodes totheir processing capability while maintaining a very high degree offlexibility. This is illustrated below with reference to FIGS. 4 and 5.

Load share values that have been calculated as described above can beaggregated to provide input to an overload control function of thesystem. To achieve this the dependency network can be augmented with aservice aggregation object that has a total work attribute whose valuedepends on the load share values of all correlators related to a givenservice in the system. The dependency function of the total workattribute simply sums all load share values together:

${{W(t)} = {\sum\limits_{i = 1}^{N}{{share}_{i}(t)}}},$where N is the number of active nodes in the load sharing group andshare_(i)(t) denotes the load share value of the ith correlator (i.e.node) at a given time.

If W(t) is less than the load balancer's sum total of share values (i.e.W_(total)), then the load sharing group cannot process the load it isexposed to and overload control should be invoked. If, on the otherhand, W(t) is more than W_(total), it means that there is spare capacityin the system. The overload control can be implemented in many ways, butthe idea is that through the overload control the number of servicerequests delivered to a node is somehow reduced.

The principles described above are illustrated in FIGS. 4 to 6.

FIG. 4 illustrates a load balancer that receives tasks is the form of anexternal load and distributes those tasks to nodes 1, 2 and 3. Initiallythe share values of all the nodes are equal, soshare_(i)(0)=W_(total)/N.

After some time, the share values are recalculated and communicated tothe load balancer. (See FIG. 5). Node1 is operating at the desired loadlevel, so there is no change in its share. Node2 has spare capacity andits share value is therefore increased. Node3 is overloaded and itsshare value is decreased. The load balancer distributes the load inproportion to the shares. The sum of shares is still greater than orequal to W_(total), so the system is performing correctly.

FIG. 6 illustrates a cluster overload situation. The shares for nodes 1and 2 are decreased, with the result that the sum of the shares is lessthan W_(total). Therefore, the cluster as a whole is overloaded.Overload control is invoked to reduce the load.

The aggregation of the load share values can be used as an indication ofthe need to increase overall processing capacity to meet the increasedload. This is a direct consequence of a prolonged need to apply overloadcontrol and can be implemented by adding an attribute to the serviceaggregation object that depends on the total work attribute of theservice aggregation object, and time. If a prolonged need to applyoverload control is detected, the system can inform the operator of theneed to add more processing capacity (i.e. nodes) to the load sharinggroup.

The nominal load value can be used in conjunction with the overloadcontrol to reach the desired level of overall processing capability(i.e. to limit the allowed overall processing capability). Over time,the system can in effect learn the correct nominal rate for a correlatorin a given node; the nominal rate can be set to be equal to the observedrate if the load share value has not been changed for some period oftime. The service aggregation object can also aggregate the rateattributes of the process groups. If the aggregation of the rateattributes is larger than the service request rate the system isdesigned to meet, and the overload control is not in use, then thesystem is able to process more work than intended. If there is a need tolimit the amount of work the system can handle, the nominal loadattributes can be decreased which will automatically start decreasingthe share values. If the aggregation of the share values falls below thelimit defined above, overload control is invoked and the system willautomatically start limiting the amount of work processed by the nodes.

In comparison to overload control mechanisms that have been implementedin the past, this approach uses information calculated by an adaptiveload balancing mechanism to implement overload control and dimensioning.One advantage of this is that the same simple information that can beused to control the adaptive load balancing function can be used asinput to overload control. The computation of the information can bedone in parallel in a distributed system.

The arrangement described above also provides a mechanism whereby anoperator can intervene to limit the total amount of processing done bythe system. This can conveniently be done by reducing the set value ofthe nominal load. This will have the effect of reducing the processingrate. This might be useful if another party had paid for a set amount ofprocessing on the system: if the system were processing at a higher ratethan the other party had paid for then the operator might want to curbthe system. To test whether the processing rate was too high theoperator could aggregate the rate attributes of the processors andcompare that aggregate with the total rate agreed with the other party.

The arrangement described above can address the problems of how to makean indication to the system's overload control of the need to startreducing the load, how to make an indication to the system (andeventually, to the operator) of the need to increase processing capacityto meet the increased load, how to dimension the system so that adesired level of overall capacity is reached and how to implement all ofthe above in a distributed fashion to increase the performance andscalability of the system

Simple overall values can be used to control the capacity of the systemas a whole and yet allow flexible configuration of the individual nodes(both software and hardware). Detailed hardware information is notneeded to control the load balancing function and the system willautomatically adjust itself to the current software and hardwareconfiguration.

The load share value can be used as an indication of a possible problemin the node, in the configuration of software executing on the node, orin the load balancing function itself. Should the load share valuebecome and remain less than a pre-set lower limit, it can be taken as anindication that a node is not able to process even the minimum amount ofwork that the load balancer can assign to it. This can happen if thehardware of the node is simply not powerful enough, the hardware is notfunctioning properly, the software processing the requests isinefficient or buggy, there is some other software on the node that isconsuming the processing capacity, or if the load balancer is notworking properly. The probable cause of the problem can be deduced ifthe system also collects CPU usage data into a CPU usage attribute ofthe processes and aggregates it to a CPU usage attribute of the processgroup using the dependency network. If the load share value of thecorrelator linked to the process group falls below the threshold but theaggregated CPU usage of the group is close to zero, it may mean thatthere are some other processes not belonging to the process group inquestion that are using up the CPU and reconfiguration of the softwaremay be in order. If, however, the CPU usage value of the process groupis large but the load share value is small it means that a small amountof work burns a lot of CPU cycles. This may be because of problems inthe software processing the requests which can be suspected if theaggregated rate of the process group is small. On the other hand, if theaggregated rate is large, the processes get a lot of service requestsfrom the load balancer although their load share should be small, whichmay indicate a problem in the load balancing algorithm. If none of theprevious conditions apply, then hardware problems may be the possiblecause of the problem.

This arrangement can be used to address the problems of how to noticethat a node cannot process the minimum load that can be assigned to it,how to utilise this as an indication of a possible problem in the nodeor in the load balancing function and how to implement it in adistributed fashion to increase the performance and scalability of thesystem

FIG. 7 illustrates a node overload situation. The sum of the shares isgreater than W_(total) but the share for node 3 has fallen below thepre-set lower limit, which in this example is taken to be 1. Indiagnosis of this problem, if the CPU usage for node 3's process groupis low then the overload might be due to a problem in the node itself(for instance due to the malfunction of hardware or other software); ifthe CPU usage for node 3 is high then the overload might be due to aproblem in the process group itself (if its rate attribute is small) orin the load balancer algorithm (if the rate attribute is large).

The object dependency network can also be applied to implementadministrative control at an appropriate and desired level. This alsoincludes the implementation of graceful shutdown behaviour for variousentities in the system.

The values of the administrative state attributes of a node, processgroup, and a process are linked together using the dependency network sothat the administrative state of the process group follows that of thenode, and the administrative state of a process follows that of theprocess group. This set-up allows the operator to control the system atan appropriate level. For example, an operator may not be interested incontrolling directly the processes that participate in the providing ofa service, but he or she might want to control whether the whole servicein a given node is available for use. This is made possible by the factthat if the operator changes the administrative state of the processgroup to locked, the dependency network automatically sets theadministrative states of the processes depending on the process group tolocked, and the processes can stop providing the service. Anotherexample is a maintenance operation to the node, where an operator mightwant to take the physical hardware out of use and replace it with newhardware. This requires that the software running on the node and alsoon other nodes be informed of the fact. This is made possible by thefact that the administrative states of all process groups on the nodedepend on the administrative state of the node, and as soon as theadministrative state of the node is changed, so are the administrativestates of all objects that depend on it.

The graceful shutdown of an entity in the system can also be implementedusing the dependency network. For example, an operator might want toexpress that a node should be taken out for maintenance gracefully, i.e.so that ongoing services on the node are allowed to be finalised beforeremoving power from the node. The shutting down value of theadministrative state attribute is propagated from the node to theprocess group, and finally to the processes themselves. As soon as theprocesses have processed all service requests to completion, they willchange their administrative states to the locked value. A reverseddependency is constructed between the processes and the process groupsuch that if the value of the administrative state of the process groupis shutting down, and if the values of the administrative states of allprocesses belonging to the process group are changed to locked, thevalue of the administrative state of the process group will also becomelocked. A similar dependency is constructed between the process groupsand the node, so that the value of the administrative state attribute ofthe node will automatically be changed to locked when all process groupsbecome locked, which means that all service requests have been processedto completion and it is now safe to turn off power without losing anyservice instances.

This is illustrated in FIGS. 8 and 9. In FIG. 8, the operator can lockthe process group and all processes whose administrative state dependson the process group are automatically locked. Likewise, the operatorcan take node X out of operation for maintenance by shutting it down andall processes will follow. In FIG. 9 the operator can take node X out ofoperation for maintenance by shutting it down gracefully and allprocesses will follow without interrupting service. When processesbecome locked, so will the process group, and ultimately the node.

The node may be configured to propagate to a control unit a messageindicating that its administrative state has been changed to locked. Inresponse to this message power to the node can be shut off safely.

It should be noted that the hierarchies and dependencies described aboveare only examples, and the actual system can have more levels ofhierarchies. Also, the dependencies can be defined in much moresophisticated ways thus allowing very complex relations to be expressed.The dependency network is a very powerful concept and lends itself tomany other uses.

The systems described above can be implemented in software or hardware.The calculations are mainly carried out by the dependency network. It ispreferred that implementation is done in a distributed fashion to makethe system more scalable. The objects that aggregate attributes of ordepend on objects in different nodes are most naturally to be placedinto a centralised manager node because they make observations of anddeductions regarding the overall system.

The load balancing and the control functions described above areindependent of each other.

One potential implementation of the invention is in a server platformthat could be used for hosting control and service layer applications(for instance CPS, HSS, SIP application server or IP RAN controllers) ina telecommunication network, especially an all IP network. The serverhardware architecture could be based on a loosely coupled network ofindividual processing entities, for example individual computers. Thiscan afford a high level of reliability and a high degree of flexibilityin configuring the platform for different applications andcapacity/performance needs. Preferably the hardware of each computernode can be based on de facto open industry standards, components andbuilding blocks. The software can be based on an operating system suchas Linux, supporting an object oriented development technology such asC++, Java or Corba. The processing entities are preferably coupled by anetwork connection, for example Ethernet, rather than via a bus. Thisfacilitates loose interconnection of the processing entities. Thearchitecture suitably comprises two computer pools: the front end IPDirectors and the server cluster. The IP Director terminates IPsec (whenneeded) and distributes service requests further to server cluster (loadbalancing). The number of IP Directors can be scaled up to tens ofcomputers and server nodes to a much larger number per installation. TheIP Director load balances the signalling traffic coming in, typicallySIP and SCTP. For SIP, load balancing is done based on call ids. ForSCTP: load balancing is done by streams inside one connection. Otherload balancing criteria can be used as well (for example based on sourceor destination addresses).

The present invention may include any feature or combination of featuresdisclosed herein either implicitly or explicitly or any generalisationthereof, irrespective of whether it relates to the presently claimedinvention. In view of the foregoing description it will be evident to aperson skilled in the art that various modifications may be made withinthe scope of the invention.

1. A method, for allocating load to processing nodes in a processingcluster arranged as an object dependency network, the method comprising:establishing at a load allocation function a plurality of loadallocation function attributes whose values are dependant on the valuesof attributes of the processing nodes, the load allocation functionattributes comprising: load ratio attributes, each load ratio attributecorresponding to a respective node and having a value indicating a loadthat has been processed by the respective node in a predetermined timeperiod as a proportion of a nominal load to be processed, rate ratioattributes, each rate ratio attribute corresponding to the respectivenode and having a value indicating a rate of requests for processingthat have been processed by the respective node in a predetermined timeperiod as a proportion of a nominal rate of requests for processing tobe processed, and load share attributes, each load share attributecorresponding to the respective node and having a value indicating aproportion of the total load on the processing cluster that is to beallocated to the respective node, wherein said each load share attributeis dependent on the load ratio attribute and the rate ratio attribute ofthe respective node; receiving requests for processing; determining towhich of the processing nodes to allocate the requests for processingbased on the load share attributes; allocating requests for processingto each processing node in accordance with the load share attribute ofthe respective node; and when the load that has been processed by therespective node changes and/or the rate of requests for processing thathave been processed by the respective node changes, automaticallyrecalculating the load share attribute of the respective node.
 2. Themethod as claimed in claim 1, wherein the determining comprisesdecreasing the load share attribute corresponding to the respective nodeby a first amount when the load ratio attribute for the respective nodeindicates that the rate of requests for processing that have beenprocessed by the respective node in a predetermined time period as aproportion of a nominal rate of requests for processing to be processedis less than a first threshold.
 3. The method as claimed in claim 2,wherein the determining comprises decreasing the load share attributecorresponding to the respective node by a second amount when the loadratio attribute for the respective node indicates that the rate ofrequest for processing that have been processed in a predetermined timeperiod as a proportion of a nominal rate of requests for processing tobe processed by the respective node is greater than the first thresholdand the rate ratio attribute for the respective indicates that the rateof requests for processing that have been processed by the respectivenode in a predetermined time period as a proportion of a nominal rate ofrequests for processing to be processed is less than a second threshold.4. The method as claimed in claim 3, wherein the first threshold isgreater than the second threshold and the first amount is greater thanthe second amount.
 5. The method as claimed in claim 1, wherein thedetermining comprises increasing the load share attribute correspondingto the respective node by a third amount when the load ratio attributefor the respective node indicates that the rate of requests forprocessing that have been processed by the respective node in apredetermined time period as a proportion of a nominal rate of requestsfor processing to be processed is greater than a third threshold and therate ratio attribute for the respective node indicates that the rate ofrequests for processing that have been processed by the respective nodein a predetermined time period as a proportion of a nominal rate ofrequests for processing to be processed is between the third thresholdand a fourth threshold.
 6. The method as claimed in claim 5, whereinotherwise the load share attribute corresponding to the node isunchanged.
 7. The method as claimed in claim 1, comprising adjusting thevalue of the nominal rate towards a currently observed rate if the loadshare attributes have remained substantially unchanged over apredetermined time period.
 8. The method as claimed in claim 1,comprising reducing the nominal load if an aggregate rate of processingof the processing nodes of the cluster is determined to be greater thana pre-set threshold.
 9. The method as claimed in claim 1, comprisingdeclaring an error state if a load share attribute corresponding to therespective node remains less than a first threshold over a predeterminedtime period.
 10. An apparatus, comprising: a plurality of processors;and a load allocation function having a plurality of load allocationfunction attributes whose values are dependant on the values ofattributes of the processors, the load allocation function attributescomprising: load ratio attributes, each load ratio attributecorresponding to a respective processor and having a value indicating aload that has been processed by the respective processor in apredetermined time period as a proportion of a nominal load to beprocessed, rate ratio attributes, each rate ratio attributecorresponding to the respective processor and having a value indicatinga rate of requests for processing that have been processed by therespective processor in a predetermined time period as a proportion of anominal rate of requests for processing to be processed, and load shareattributes, each load share attribute corresponding to the respectiveprocessor and having a value indicating a proportion of the total loadon a cluster that is to be allocated to the respective processor,wherein said each load share attribute is dependent on the load ratioattribute and the rate ratio attribute of the respective processor,wherein the load allocation function is configured to receive requestsfor processing, determine to which of the processors to allocate therequests for processing based on the load share attributes, and allocaterequests for processing to each processor in accordance with the loadshare attribute of the respective processor, and wherein the loadallocation function is configured, when the load that has been processedby the respective processor changes and/or the rate of requests forprocessing that have been processed by the respective processor changes,to automatically recalculate the load share attribute of the respectiveprocessor.
 11. An apparatus having a processor, the apparatuscomprising: processing means for processing; load balancing means forbalancing, the load balancing means having a plurality of loadallocation function attributes whose values are dependant on the valuesof attributes of the processing means, the load allocation functionattributes comprising: load ratio attributes, each load ratio attributecorresponding to a respective processing means and having a valueindicating a load that has been processed by the respective processingmeans in a predetermined time period as a proportion of a nominal loadto be processed, rate ratio attributes, each rate ratio attributecorresponding to the respective processing means and having a valueindicating a rate of requests for processing that have been processed bythe respective processing means in a predetermined time period as aproportion of a nominal rate of requests for processing to be processed,and load share attributes, each load share attribute corresponding tothe respective processing means and having a value indicating aproportion of a total load on a cluster that is to be allocated to therespective processing means, wherein said each load share attribute isdependent on the load ratio attribute and the rate ratio attribute ofthe respective processing means, wherein the load balancing meansreceives requests for processing, wherein the load balancing meansdetermines to which of the processing means to allocate the requests forprocessing based on the load share attributes, wherein the loadbalancing means allocates requests for processing to each processingmeans in accordance with the load share attribute, of the respectivenode processing means, and wherein the load balancing means, when theload that has been processed by the respective processing means changesand/or the rate requests for processing that have been processed by therespective processing means changes, automatically recalculates the loadshare attribute of the processing means.